Vision Based Registration for Augmented Reality Using Multi-planes in Arbitrary Position and Pose by Moving Uncalibrated Camera

This paper presents a novel AR registration method for overlaying a virtual object onto real world by multiple planes in arbitrary positions and poses. In this method, we do not require neither artificial markers nor sensors, but natural feature points on arbitrary multiple planes. We assign a 3D coordinate system for each plane independently. For each coordinate system, the projection matrix is computed in order to relate the coordinate system to the input images. Then, all the 3D coordinates are integrated into a projective 3D coordinate space that is defined by two reference images. Such integration can reduce errors in registration of a virtual object with the real world coordinate. For demonstrating the effectiveness of the proposed method, some experiments for overlay of virtual objects into image sequences taken with a handy video camera are performed. Accuracy evaluation by the number of planes via a synthesized image sequence is also performed, which indicates that the integration of a number of planes reduces the registration error. Additionally, comparison of proposed method with one of related work is performed. Our approach can contribute to image generation applicable to movie, sports casting, TV show, and so on.